Lecture-14 : Embedded Markov Chain and Holding Times

Lecture-14 : Embedded Markov Chain and Holding Times

Proof. It is clear that {Sn 6 t} is completely determined by the history Ft = σ(X(u), u 6 t) until time t. Lemma 1.2. For a homogeneous CTMC, each sojourn time Tn is a continuous memoryless random variable, and the sequence of sojourn times (Tj : j > n) is independent of the past FSn−1 conditioned on Xn−1. Proof. We observe that the sojourn time Tn equals the excess time Y (Sn−1) in state X(Sn−...

Lecture 3: Continuous times Markov Chains. Poisson Process. Birth and Death Process. 1 Continuous Time Markov Chains

In this lecture we will discuss Markov Chains in continuous time. Continuous time Markov Chains are used to represent population growth, epidemics, queueing models, reliability of mechanical systems, etc. In Continuous time Markov Process, the time is perturbed by exponentially distributed holding times in each state while the succession of states visited still follows a discrete time Markov ch...

Introduction to Computational Biology Lecture # 14: MCMC - Markov Chain Monte Carlo

Time and Ratio Expected Average Cost Optimality for Semi-Markov Control Processes on Borel Spaces

We deal with semi-Markov control models with Borel state and control spaces, and unbounded cost functions under the ratio and the time expected average cost criteria. Under suitable growth conditions on the costs and the mean holding times together with stability conditions on the embedded Markov chains, we show the following facts: (i) the ratio and the time average costs coincide in the class...

Lecture 14: October 22 14.1 Relating Mixing times Using Flows 14.1.1 Theory

This section describes a flow-based method for bounding the Poincaré constant (or, in the reversible case, the spectral gap) of a Markov chain based on that of another Markov chain, due to Diaconis and Saloff-Coste [DS93]. Assume we are given two ergodic, lazy Markov chains P and P̃ which share the same stationary distribution π over some state space Ω. Furthermore, assume that we have already p...